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1.  Introduction 


An  innovative  sensor  suit  is  developed,  which  can  be  conveniently  put  on  by  an  operator  to  detect  his  or  her  motion  intention  by 
non-invasively  monitoring  his  or  her  muscle  conditions  such  as  the  shape,  the  stiffness  and  the  density.  This  sensor  suit  is  made  of 
soft  and  elastic  fabrics  embedded  with  arrays  of  MEMS  sensors  such  as  muscle  stiffness  sensor,  ultrasonic  sensors,  accelerometers  and 
optical  fiver  sensors,  to  measure  different  kinds  of  human  muscle  conditions.  In  this  project  period,  we  successfully  developed  the 
sensor  suit,  which  is  based  on  EMG  sensors.  Moreover,  the  various  technologies  we  developed  in  this  project  such  as  a  MEMS 
accelerometer,  a  fiber  optic  sensor,  a  muscle  stiffness  sensor  and  muscle  density  sensor  are  evaluated  those  applicability  of  the 
measurement  of  human  body  condition  through  various  experiment.  Now,  these  sensors  we  developed  are  introduced  various 
application  field,  and  achieve  a  successful  outcome.  The  details  and  the  experimental  results  about  our  sensor  suit  and  sensors  we 
developed  in  this  project  are  shown  below. 

2.  Muscle  Stiffness  Sensor  and  Its  Application 

2.1  Muscle  stiffness  sensor 

An  innovative  sensor  is  developed  as  a  man-machine  interface  to  detect  operator’s  intention  for  controlling  the  assisting  device. 
The  sensor  detects  stiffness  information  form  a  target  muscle.  The  activation  level  of  the  muscle  can  be  estimated  from  the  obtained 
stiffness  data  of  the  muscle  because  the  muscles  always  get  stiff  when  they  act.  The  muscle  activation  obtained  from  the  sensor  is 
used  to  derive  a  command  signal  thus  the  operator  can  maneuver  the  mechanical  glove  with  desired  force  at  proper  timing.  It  would 
be  the  best  if  the  muscle  of  the  operator’s  forefinger  were  still  active  and  available  as  command  muscle.  However,  muscles  of  the 
operator’s  fingers  are  completely  inactive.  Thus  the  muscle,  Flexor  Carpi  Ulnaris,  that  is  originally  for  wrist  flexing  is  used  as  the 
command  muscle.  Figure  2.1  shows  the  muscle  stiffness  sensor.  This  sensor  has  two  components,  a  flat  disk  (32mm  in  diameter  and 
12mm  in  thickness)  and  a  button  (6mm  in  diameter  and  4mm  in  height)  in  the  center  of  the  disk.  The  whole  weight  of  the  sensor 
without  the  cable  is  34g.  This  sensor  is  attached  on  the  target  muscle  and  tightened  by  a  belt.  Two  small  pressure  sensors  are  used  to 
measure  the  force  loaded  on  the  button  and  the  force  on  the  entire  sensor. 


Fig.2.1  Muscle  stiffness  sensor 


2.2  Device  for  Assisting  Grasping  Function 

To  evaluate  the  applicability  of  our  muscle  stiffness  sensor  for  sensor  suit,  A  wearable  device  is  developed  to  recover  the  patient’s 
fingering  function.  This  device  is  put  on  operator’s  hand  and  helps  his  fingering  function.  Operator’s  forefinger  is  guided  toward  his 
fixed  thumb  by  the  mechanical  glove.  The  forefinger  is  actuated  by  an  air  cylinder  based  on  an  operator’s  command  signal.  An 
unique  sensor  is  developed  in  this  study  and  used  as  a  man-machine  interface  to  control  the  device.  The  sensor  is  attached  to  one  of 
his  sound  muscles  called  Flexor  Carpi  Ulnaris  for  wrist  flexion.  He  can  comfortably  control  its  activation  level.  The  muscle  is  called 
as  command  muscle  since  he  can  operate  the  assisting  device  through  the  activation  of  the  muscle.  The  sensor  detects  the  muscle 
activation  level  non-invasively.  Based  on  the  muscle  activation  level,  the  grasping  timing  and  force  are  determined.  It  is  confirmed 
through  experiments  that  the  patient  can  pick  up  small  daily  necessaries  such  as  a  pen  and  a  cup  with  his  finger  by  using  this  assisting 
device. 

2.3  System  configuration 

Figure  2.2  shows  the  configuration  of  the  assisting  device.  This  device  is  composed  of  a  mechanical  glove,  a  muscle  stiffness 
sensor,  an  air  control  unit  and  a  computer.  The  operator  sends  command  signals  to  the  controller  through  the  muscle  stiffness  sensor 


attached  on  his  arm.  The  controller  regulates  air  pressure  inside  the  air  cylinder  of  mechanical  glove  based  on  the  command  signals  so 
that  the  fingering  operation  is  achieved  based  operator’s  intention. 


Fig.2.2  System  configuration 


2.4  Mechanical  glove 

Figure  2.3  and  Figure  2.4  show  the  developed  mechanical  glove  and  its  structural  sketch  respectively.  The  mechanical  glove  is 
composed  of  a  finger  frame  to  flex  a  forefinger  and  a  base  to  be  attached  on  the  hand  and  an  air  cylinder  mounted  on  the  base.  The 
finger  frame  consists  of  three  links,  link  1,  link  2  and  link  3  as  shown  in  the  Fig.3.  Each  link  is  connected  by  small  sub-links  each 
other.  All  joints  of  the  finger  fame  are  driven  simultaneously  with  the  actuation  of  an  air  cylinder.  The  total  weight  of  the  mechanical 
glove  is  1 10[g]. 


Fig.  2.3  Mechanical  glove 
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2.5  Control  system 

Figure  2.5  shows  the  configuration  of  the  control  system.  A  personal  computer  (PC)  sends  and  receives  signals  through  a  Field 
Programmable  Gate  Array  (FPGA)  to  control  the  air  cylinder  of  the  mechanical  glove.  Muscle  stiffness  information  and  air  pressure 
inside  the  air  cylinder  are  fed  to  the  PC  through  a  signal  conditioning  circuit.  FPGA  generates  Pulse  Width  Modulation  (PWM) 
signals  to  control  solenoid  valves.  A  pair  of  solenoid  valves  is  used  to  control  air  pressure  of  each  chamber  of  the  air  cylinder.  One  is 
for  pressuring  and  the  other  is  for  exhausting.  Air  pressure  inside  the  both  chambers  is  always  monitored  with  air  pressure  sensors.  A 
duty  ratio  of  PWM  signal  is  decided  to  control  each  valve  so  that  the  pressure  inside  the  air  cylinder  always  follows  the  reference 
pressure.  The  reference  pressure  is  given  by  the  equation  (1). 


Prf  —  Pb  ~  +  K  x  S 

where, 

:  Reference  pressure  of  chamber  for  extending 
Ppj :  Reference  pressure  of  chamber  for  flexing 
Pg  :  Base  pressure 

Pop  :  Offset  pressure  for  extending  finger  under  no 
command  signals 
K :  C  ontrol  gain 

S  :  Muscle  stiffness  parameter 
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Fig.2.5  Control  system 


2.6  Experiment 

The  accuracy  of  the  pinching  force  is  compared  between  the  case  of  the  patient  with  the  assisting  device  and  the  case  of  non¬ 
disabled  examinees  with  their  own  ability.  Before  the  maneuverability  evaluation,  the  basic  property  of  the  assisting  device  is 
examined.  The  pinching  force  of  the  mechanical  glove  is  recorded  under  stepwise  input  signal.  Eq.(l)  shows  the  control  algorithm  of 
the  assisting  device.  The  mechanical  glove  is  kept  opened  under  the  condition  that  the  commandmuscle  is  relaxed.  As  the  command 
muscle  is  activated,  the  pressure  inside  the  cylinder  chamber  for  flexing  finger  is  increased.  Fig. 6  shows  the  pinching  force  measured 
under  a  stepwise  input  signal.  The  maximum  pinching  force  is  about  2.3  [N].  This  limitation  of  pinching  power  comes  from  the 
capability  of  air  compressor.  There  are  also  limitations  of  increasing/decreasing  ratio  of  pinching  force.  The  maximum  increasing  and 
decreasing  ratio  of  the  pinching  force  is  about  0.5 [N/s]  and  0.75 [N/s]  respectively.  This  limitation  comes  from  the  capability  of 
solenoid  valves.  Based  on  these  limitations,  the  amplitude  and  the  frequency  of  the  reference  pinching  force  are  decided  for  the 
maneuverability  evaluation.  The  patient  introduced  at  the  beginning  of  this  paper  and  3  non-disabled  examinees  take  part  in  this 
experiment.  All  examinees  are  male.  Examinee  A,  examinee  B  and  examinee  C  are  30,  32  and  22  year  old  respectively.  In  the 
experiment,  examinees  pinch  a  pressure  sensor  with  their  forefinger  and  thumb.  The  pinching  force  measured  by  the  pressure  sensor 
is  displayed  on  the  computer  screen.  A  reference  pinching  force  that  is  sinusoidal  curve  with  amplitude  0.7 [N]  and  randomly  variable 
frequency,  0.1 8-0.25 [Hz]  is  also  displayed  on  the  screen.  The  examinee  is  required  to  control  the  pinching  force  to  be  equal  to  the 
reference  force  for  1 5  second  during  the  experiment.  The  Root  Mean  Square  Error  (RMSE)  between  the  measured  pinching  force  and 
the  reference  pinching  force  is  recorded  as  a  score  to  evaluate  the  maneuverability.  The  Experiment  is  conducted  for  three  cases  as 
shown  in  Figure  2.6. 


Case  1  Fingering  operation  of  patient  Case  2  Fingering  operation  of  non-  Case  3  Fingering  operation  of  non- 
with  assisting  device  disabled  examinee  disabled  examinee  with  assisting  device 


Fig.2.6  Experimental  setup 

In  the  Case3,  the  mechanical  glove  is  tested  on  a  bench  because  it  is  found  out  that  finger  moves  unconsciously  when  the  non¬ 
disabled  examinee  try  to  operate  the  mechanical  glove.  A  wooden  thumb  is  attached  to  the  mechanical  glove  to  set  a  pressure  sensor 
on  it  instead  of  the  operator’s  finger  so  that  pinching  force  can  be  measured  without  unintentional  affect  of  examinee’s  muscular 
function. 

2.7  Experimental  result 

Figure  2.7  shows  patient’s  pinching  force  measured  in  the  experiment  of  Casel.  It  is  clear  that  the  patient  succeeded  to  control  the 
assisting  device  to  match  the  pinching  force  to  the  reference  force  displayed  on  the  computer  screen.  Figure  2.8  and  Figure  2.9  show 
examinee  A’s  pinching  force  measured  in  the  experiment  of  Case2  (without  assisting  device)  and  Case3  (with  assisting  device) 
respectively.  The  data  of  Case2  are  rather  smoother  than  that  of  Case3  however  no  remarkable  difference  can  be  recognized  in  the 


amount  of  error  under  finger  force  control  between  these  two  cases.  The  patient’s  scores  obtained  through  five  measurements  and  the 
average  value  are  shown  in  Figure  2.10.  Learning  effect  can  be  recognized  at  the  first  part  of  the  measurements.  Figure  2.11  shows 
the  examinee  A’s  score  of  five  measurements  in  both  Case2  &  Case3.  It  can  be  seen  that  the  level  and  the  tendency  of  the  error  is  quit 
similar  between  these  three  data  in  Figure  2.10  and  Figure  2.1 1.  Similarly,  the  same  measurement  is  repeated  for  the  examinee  B  and 
examinee  C.  The  average  score  of  all  the  examinees  are  shown  in  Figure  2.12.  By  the  help  of  assisting  device  the  patient’s  score 
becomes  similar  to  that  of  non-disabled  examinees.  The  assisting  device  successfully  recovers  patient’s  fingering  function  and 
achieves  the  similar  level  of  dexterity  on  controlling  the  finger  force. 
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Fig.2.7  Experimental  data  of  patient  (case  1) 
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Fig.2.8  Experimental  data  of  examinee  A  (case  2) 


Fig.2.10  Score  (case  1) 
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Fig.2.1 1  Score  of  examinee  A  (case  2  &  case  3) 


Fig.2.9  Experimental  data  of  examinee  A  (case  3) 
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Fig.2.1 2  Five  times  average  score  of  all 
examinee  (case  1,  2  and  3) 


2.8  Summary 

A  wearable  device  that  assists  fingering  function  is  developed  for  a  disabled  patient  and  its  maneuverability  is  experimentally 
evaluated.  Dexterity  on  controlling  fingering  force  is  evaluated  in  the  following  three  cases.  In  the  first  case  a  patient  who  completely 
disabled  fingering  functions  operated  his  finger  by  the  help  of  the  assisting  device.  In  the  second  case  non-disabled  examinees 
operated  their  finger  by  their  own  ability.  In  the  third  case  non-disabled  examinees  operated  their  finger  indirectly  by  using  the 
assisting  device.  No  remarkable  difference  can  be  recognized  in  dexterity  of  fingering  operation  between  these  three  cases.  It  can  be 
said  that  the  assisting  device  developed  in  this  study  recovers  fingering  function  of  disabled  with  the  equivalent  level  to  that  of 
ordinary  people  on  finger  force  control.  It  is  demonstrated  that  the  assisting  device  has  enough  maneuverability  for  its  control  through 
the  experiments. 


3.  MEMS  Accelerometer  and  Fiber  Optic  Sensor 


To  measure  the  position  and  motion  of  a  human’s  body,  we  developed  sensing  systems  using  MEMS  accelerometers  and  optical 
fiber  sensors  shown  in  Figure  3.1.  The  accelerometer  can  detect  the  angular  displacement  by  using  the  gravity  acceleration,  and  the 
optical  fiber  sensor  can  detect  the  bending  angle  of  a  joint.  Figure  3.2  (a)  and  (b)  respectively  show  the  angles  measured  by  the 
accelerometer  and  the  optical  fiber  sensors  compared  with  the  actual  angles.  Each  measurement  is  repeated  several  times.  Excellent 
repeatability  and  measurement  accuracy  are  observed.  Compared  with  the  accelerometers,  the  optical  fiber  sensors  are  more  accurate 
and  thus  are  recommended  for  measuring  the  major  joints  of  a  human  body.  On  the  other  hand,  the  accelerometer  is  small  in  size  and 
thus  easy  to  monitor  the  entire  human  body. 


(a)  Accelerometer 


(b)  Optical  fiber  sensor 


Fig. 3 . 1  Sensors  for  measuring  body  position 
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(a)  Angle  by  the  optical  fiber  sensor 


Fig.3.2  Experimental  result  of  each  sensor 


4.  EMG  Sensor 

4.1  Muscle  stiffness  estimation  using  EMG  sensor 

An  EMG  is  a  microscopic  electrical  charge,  which  is  generated  when  a  muscle  contracts.  By  detecting  this  electrical  charge,  the 
muscle  activity  can  be  estimated.  An  EMG  sensor  is  embedded  in  our  sensor  suit,  as  shown  Figure  4.1,  to  detect  the  electrical  charge. 
Figure  4.2  shows  the  correlation  between  the  muscle  stiffness  obtained  from  a  muscle  stiffness  sensor  (developed  earlier  in  this 
project)  and  the  EMG  obtained  from  EMG  sensor. 


Fig.4.1  EMG  sensor 


Fig.4.2  Correlation  between  EMG  and  muscle  stiffness 


4.2  Estimation  of  dynamic  torque  of  elbow  joint  using  reflective  ultrasonic  muscle  activity  sensor 

We  developed  an  auto-adaptive  calibration  system  using  EMG  sensors  and  experimentally  confirmed  its  effectiveness.  During  this 
year,  we  developed  an  auto-adaptive  system  based  on  neural  networks  using  muscle  activity  sensors.  We  developed  a  reflective 
ultrasonic  sensor  as  the  muscle  activity  sensor.  By  using  this  sensor,  both  the  reflected  ultrasound  and  the  transmitted  ultrasound  can 
be  detected  at  same  time.  As  a  result,  the  accuracy  is  improved  dramatically  and  it  becomes  possible  to  measure  the  activities  of  the 
muscles  behind  bones. 

Figure  4.3  shows  the  developed  sensor  cuff  embedded  with  7  ultrasonic  sensors.  This  sensor  cuff  can  detect  the  reflected  ultrasound 
sound  and  transmitted  ultrasound  sound  at  the  same  time.  Figure  4.4  shows  the  reflected  ultrasound  (R1  and  R2)  and  transmitted 
ultrasound  (R3  and  R4)  when  the  ultrasound  is  transmitted  from  the  transmitter  Tl.  Figure  4.5  shows  the  actual  ultrasound  waveform 
obtained  from  the  receivers  Rl,  R2,  R3  and  R4. 


Fig.4.8  Reflective  ultrasonic  sensors 
embedded  in  small  cuff 


Fig.4.4  Reflected  ultrasonic  sound  and  transmitted  ultrasonic  sound 
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Fig.4.5  Waveform  of  transmitting  (Tl)  and  receiving  (R1-R4)  ultrasound. 
(Waves  are  added  offset  for  plotting) 


4.3  Experiment  and  result 

Figure  4.6  shows  the  experimental  setup  for  estimating  the  torque  of  human  arms  based  on  the  sensor  data  from  the 
sensor  cuff.  To  measure  the  actual  torque  for  the  reference,  we  used  the  torque  measurement  equipment  as  shown  in 
Figure  4.6.  By  an  examinee  grabs  a  handle  attached  on  its  tip,  the  bending  movement  of  his/her  arm  and  the  joint 
movement  of  the  torque  measurement  equipment  are  synchronized.  At  this  time,  the  actual  torque  of  elbow  joint  is 
obtained  by  a  torque  sensor  attached  on  the  joint  of  this  equipment.  The  examinee  wears  the  sensor  cuff  on  his/her  arm  to 
estimate  the  joint  torque,  and  then  attaches  torque  measurement  equipment  to  measure  actual  torque  as  the  reference 
values.  For  the  learning  purpose,  the  examinee  follows  the  pre-specified  elbow  bending  motion  instruction  displayed  on  a 
monitor  to  moves  his/her  arm.  From  measured  sensor  data,  the  arm  torques  of  the  examinee  are  estimated  dynamically 
using  a  pre-designed  neural  networks.  Figures  4.7  and  4.8  show  the  experimental  result.  In  this  figure,  x  shows  the  actual 
torque  measured  by  the  torque  measurement  equipment,  xM  shows  the  torque  estimated  by  multiple  regression  analysis 
and  xN  shows  the  torque  estimated  by  the  neural  network.  We  performed  7  experiments  (2nd  to  8th)  after  1  learning 
motion  (1st)  as  shown  in  Figure  4.7.  As  shown  in  this  figure,  the  estimated  torque  agree  with  the  measured  torque  with 
high  accuracy. 


Fig.4.6  Experimental  setup 
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Fig.4.7  Estimated  and  measured  torque 
(xM: Estimation  by  multiple  regression  analysis) 
(xN: Estimation  by  neural  network) 
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Fig.4.8  Comparison  of  estimated  and  measured  torques 
(xM: Estimation  by  multiple  regression  analysis,  xN:Estimation  by  neural  network) 


5.  3D  Measurement  and  Imaging  Processing  System  for  Detection  Muscle  Position  and 
Boundary 

A  3D  measurement  and  imaging  processing  system  for  detection  human  muscle  shape  has  been  developed.  A  human  muscle 
boundary  analysis  is  one  of  the  key  issues  for  our  sensor  suit.  These  kinds  of  information  measured  by  this  3D  measurement  system  is 
used  for  estimation  of  optimal  sensor  position  ant  it  makes  our  sensor  suit  more  accurate  and  reliable.  Especially,  the  analysis  of 
shifting  of  human  muscle  boundary  responding  to  operator’s  body  movement  is  important  information  to  estimate  optimal  sensor 
positions.  By  using  our  system,  the  human  muscle  boundary  lines  drawn  by  an  expert  of  surface  anatomy  to  examinee’s  body  and 
examinee’s  body  shape  can  be  obtained  as  a  3D  model  data  accurately  and  quickly.  Furthermore  a  database  of  muscle  boundary  lines 
can  be  created  at  short  times. 

5.1  System  configuration 

A  developed  system  consists  of  a  LCD  projector,  a  digital  camera  and  a  computer  as  shown  in  Figure  5.1.  A  LCD  projector 
projects  a  mesh  pattern  to  examinee’s  body  and  muscle  boundary  lines,  these  are  drawn  by  an  expert  of  surface  anatomy  with  a  bright 
pen.  A  digital  camera  captures  this  body  image  and  sends  it  to  a  computer.  On  this  computer,  an  image  processing  is  performed  and  a 
2-dimentional  image  is  translated  to  3D  polygon  information.  Furthermore,  the  boundary  lines  also  detected  as  textures  and 
vectorized  lines  through  this  image  processing. 


Fig.5.1  System  configuration 


5.2  3D  measurement 


If  the  corresponding  intersection  points  of  a  mesh  pattern  in  a  captured  2D  image  to  intersection  points  projected  by  a  projector  are 
found  out,  3D  position  of  each  intersection  points  can  be  easily  obtained  by  triangulation  method. 

5.3  Image  processing 

The  captured  image  by  a  digital  camera  is  sent  to  and  manipulated  on  a  computer.  A  projected  mesh  pattern  in  captured  image  is 
converted  a  vectorized  mesh  model  at  short  times  through  the  following  image  processing  procedure. 

(a)  Binarization 

Binarization  is  performed  with  an  optimal  threshold  obtained  by  Ohtsu’s  method.  A  obtained  histograms  from  a  captured 
images  is  clearly  divided  into  low  intensity  pixels  and  high  intensity  pixels,  and  tend  to  be  diphasic,  therefore  the  Ohtsu’s 
method  which  can  find  the  dale  in  the  diphasic  histogram  as  a  threshold  is  suited  for  our  purpose  as  shown  in  Figure  5.2. 


Grayscale  image  Binarized  image 


Fig. 5. 2  Binarization 


(b)  Thinning  for  detection  of  mesh  pattern 

As  a  preprocessing  for  finding  intersection  points  in  a  captured  image,  a  thinning  technique  is  applied.  As  shown  in  Figure 
5.3,  the  thick  lines  are  translated  one  pixel  thickness  lines  by  grinding  down  recrementitious  pixels. 


Binarized  image  image  after  thinning 

Fig. 5. 3  Thinning 


(c)  Detection  of  intersection  points  of  mesh  pattern 

To  find  vertex  of  polygons,  intersection  points  are  detected  by  surveying  pixels  laying  side  by  side  of  each  pixels. 
Furthermore,  to  avoid  detection  of  discarded  intersection  points,  those  are  generated  by  the  error  in  thinning  process,  the 
intersection  points  those  placed  near  each  ether  are  merged  as  one  intersection  point  as  shown  in  Figure  5.4. 


Fig.5.4  Detection  of  intersection  points 


(d)  Detection  of  muscle  boundary  lines  drawn  by  an  expert  of  surface  anatomy 

The  boundary  lines  drawn  by  an  expert  of  surface  anatomy  is  detected  by  binarization  process,  and  put  it  on  as  a  texture 
using  texture-mapping  technique.  Furthermore,  using  image  processing  through  (a)  to  (c)  above,  the  muscle  boundaries  also 
stored  as  vectorized  information. 


5.4  Experiment  and  result 

An  experiment  to  evaluate  applicability  of  developed  3D  measurement  system  for  our  sensor  suit  is  performed.  In  this  experiment, 
we  focused  on  the  muscle  boundary  lines  of  human  legs.  Figure  5.5  shows  examinee’s  leg  image  with  the  muscle  boundary  lines 
drawn  by  an  expert  of  surface  anatomy.  Background  of  the  examinee  is  covered  with  black  drape  to  avoid  false  detection  of  image 
processing. 


Figure  5.6  shows  the  detected  3D  polygon  mesh  by  capturing  a  projected  grid  pattern.  As  shown  in  this  figure,  the  shape  of 
examinee’s  leg  is  transformed  3D  polygon  model  correctly.  Because  of  the  short  measurement  time,  it  is  possible  to  capture  the  leg 
image  and  transform  it  to  3D  polygon  mesh  continuously  to  generate  animation  data  as  shown  Figure  5.7. 


Fig. 5. 6  Measured  3D  polygon  mesh 


Fig. 5. 7  Creation  of  animation 


Figure  5.8  shows  a  detected  corresponding  muscle  boundary  lines  drawn  by  an  expert  of  surface  anatomy.  As  shown  in  this  figure, 
the  boundary  lines  also  clearly  detected  and  translated  to  vectrized  information. 


Photograph  with  Vectorized 

Fig. 5. 8  Creation  of  animation 

5.5  Summary 

3D  Measurement  and  imaging  processing  system  for  detection  muscle  position  and  boundary  has  been  developed.  Using  developed 
system,  a  database  on  human  motions  and  corresponding  muscle  boundary  lines  drawn  by  an  expert  of  surface  anatomy  can  be  quickly 
established.  Based  on  such  a  database,  the  optimal  numbers  and  locations  of  sensors  on  the  sensor  suit  can  be  effectively  determined. 


6.  Sensor  Network 

6.1  sensor  network  using  microcomputer 

The  increase  of  sensors  causes  the  increase  of  electric  wires  in  the  suit,  which  decrease  the  robustness  of  the  data  transmission.  To 
cope  with  this  problem,  we  have  developed  a  sensor  network  with  minimum  lengths  of  electric  wires.  As  illustrated  in  Figure  6.1,  the 
developed  sensor  network  has  a  much  simpler  network  configurations  and  thus  less  susceptible  to  damage. 
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Fig. 6.1  Sensor  Network 


6.2  Prototype  of  sensor  network 

Experimental  sensor  networks  were  fabricated.  Figure  6.2  shows  a  sample  of  the  accelerometer  network.  A  routing  algorithm  used 
for  the  Internet  connection  is  applied  for  this  network.  Even  if  some  parts  of  the  network  suffer  from  damage,  the  sensor  network  still 
can  works  properly  to  transmit  all  the  sensor  data  to  the  controller.  Figure  6.3  illustrates  how  the  sensor  network  works  after  a  portion 
of  the  network  is  damaged. 


Fig. 6. 2  Prototyped  network  of  accelerometer 
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Fig.6.3  Sketch  of  the  decentralized  network  connection 


6.3  Simulation 

This  failure-tolerant  feature  of  the  sensor  network  is  confirmed  through  the  computer  simulation.  The  virtual  network  on  the 
computer  screen  is  shown  in  Figure  6.4.  Excellent  sensing  abilities  as  well  as  high  reliability  of  the  sensor  network  system  are 
demonstrated  through  both  experiments  and  computer  simulations. 
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Fig. 6. 4  Computer  simulation  of  the  decentralized  network  connection 


6.4  Experiment  and  result 

To  evaluate  our  system,  we  have  tested  our  sensor  network  system  with  small  sized  sample  network  as  shown  Figure  6.5  and 
computer  simulation  software.  Each  node  has  a  microcomputer  and  proper  routing  table  stored  in  its  computer.  Each  node  has  also 
LED  on  its  main  board  so  that  we  can  check  the  packet  position  and  movement  with  eyes.  In  this  network,  a  packet  is  sent  from  node 
3,  and  the  destination  of  the  packet  is  node  0. 


Fig. 6. 5  Sensor  network  for  experiment 


Figure  6.6  shows  the  actual  movement  of  data  packet  under  condition  1  and  condition  2.  Condition  1  is  assumed  that  connection 
cables  are  not  damaged,  and  conndition2  is  assumed  that  the  connection  cable,  which  connects  node  3  and  node  2  is  damaged.  As 
shown  in  Figure  6.6,  In  case  of  condition  1,  the  packet  sent  from  node  3  is  transmitted  through  the  optimal  path  according  to  the 


routing  table.  In  case  of  condition  2,  the  connection  cable,  which  connects  node  3  and  node  1  is  damaged.  However,  the  packet  sent 
form  node  3  is  correctly  transmitted  to  the  destination  node  0  with  avoiding  the  damaged  connection. 
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Fig. 6. 6  Experimental  result 


6.5  Summary 

A  sensor  network  system  has  been  developed.  The  developed  sensor  network  has  a  much  simpler  network  configurations  and  thus 
less  susceptible  to  damage.  Furthermore,  the  computer  simulation  software  also  has  been  developed.  We  successfully  confirmed  the 
efficiency  of  our  sensor  network  system  through  the  experiment  using  actual  small  network  and  simulation  software. 


7.  Auto-Calibration  System  for  EMG  Sensor  Suit 


An  innovative  haptic  sensor  suit  with  a  distributed  sensing  capacity  is  being  developed  based  on  the  fact  that  human  motion  is 
actuated  by  certain  muscles,  and  thus  the  shape,  stiffness  and  density  of  the  muscles  change  accordingly  with  the  motion  and  loading 
conditions  of  the  human  body.  The  proposed  suits  are  made  of  soft  elastic  fabrics  in  which  arrays  of  tiny  sensor  disks  and  optical 
fibers  are  embedded.  The  sensor  disks  are  made  of  strain  gauges  and  ultrasonic  sensors  for  measuring  stiffness  and  density  of  the 
muscles.  The  different  types  of  sensor  are  integrated  in  the  sensor  suit  and  the  data  from  all  these  different  sensors  are  fused  to 
achieve  accurate  and  reliable  measurement  of  the  muscle  conditions  and  motion  and  furthermore  reliable  estimate  of  torques  of  the 
operator.  This  estimated  torque  can  be  used  to  accurately  control  the  power  amplifier  in  a  timely  manner.  In  our  project,  an 
innovative  haptic  sensing  system  with  a  distributed  sensing  capacity  is  proposed.  We  introduced  tile  electrode,  in  order  to  cope  with 
the  problem.  The  experiment,  which  presumes  torque  from  line  potential  using  EMG  sensor  suit,  was  carried  out  to  examine  the 
validity  of  the  system. 


7.1  EMG  Sensor  Suit 

Developed  auto-calibration  system  is  focused  on  EMG  sensors.  Because,  EMG  is  the  most  effective  sensor,  which  can  perform  the 
biometry.  However,  it  is  difficult  to  equip  the  place  same  each  time  with  a  sensor  disk,  and  sensor  disk  will  move  during  wearing. 
The  elasticity  in  which  a  suit  has  these  problems  is  the  cause.  To  measure  the  accurate  EMG,  the  sensors  embedded  in  our  suit  should 
focus  on  the  aggregate  of  neuromuscular  junction  called  innervation  zone.  To  cope  with  this  problem,  a  tile-like  sensor  disk  has  been 
developed  as  shown  Figure  7.1.  The  EMG  sensors,  which  detects  EMG  is  shifted  according  to  the  position  of  innervation  zone  as 
shown  Figure  7.2. 


Fig.7.1  Tile-like  sensor  disk 
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Fig.7.2  Division  method  of  tile  electrode 


7.2  Detection  of  innervation  zone 

The  position  of  innervation  zone  can  be  estimated  by  using  correlation  coefficient  of  each  waveform  obtained  from  EMG  sensors. 
The  line  EMG  waveform  actually  measured  with  the  array  electrode  of  three  channels  is  shown  in  Figure  7.2.  From  this,  it  can  check 


that  correlation  is  in  the  waveform  of  Ch.l  and  Ch.2.  Moreover,  the  waveform  of  Ch.2  and  Ch.3  can  check  that  it  is  also  reversed. 
We  describe  how  to  actually  presume  the  position  of  innervation  zone  using  an  array  electrode.  As  shown  in  Figure  7.3,  when  the 
array  electrode  of  three  channels  is  used,  the  position  of  five  innervation  zone  can  be  considered.  The  tendency  of  the  position  of  the 
nerve  rule  belt  and  the  correlation  coefficient  of  each  channel  was  shown  in  Table  1.  Moreover,  Co.l  was  set  to  the  correlation 
coefficient  between  Ch.l  and  Ch.2.  Co. 2  was  set  to  the  correlation  coefficient  between  Ch.l  and  Ch.3.  Co. 3  was  set  to  the  correlation 
coefficient  between  Ch.2  and  Ch.3. 


Innervation  zone 


Fig. 7. 3  Relation  between  innervation  zone 
and  EMG  sensors 


Fig.7.4  EMG  signal  influenced  of  shell 
innervation  zone 


Table  1  Relation  between  position  of  second  and  correlation  of  each  channel 
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7.3  Experiment  and  experimental  result 

An  experiment  of  a  joint  torque  presumption  from  tile  EMG  electrode  was  performed.  In  this  experiment,  knee  joint  torque  is 
presumed  from  tile  electrode  attached  on  examinee’s  thigh  as  shown  in  Figure  7. 5 -(a).  The  detected  EMG  signal  is  amplified  10000 
times  with  amplifier  (amplifier  NIHON  KOUDEN  MME-3 116)  and  5Hz  highpath  filter  is  applied.  Simultaneously,  joint  torque  is 
measured  by  a  torque-measuring  (Figure  7. 5 -(b))  instrument.  Teaching  data  was  measured  using  the  tile  electrode  arrayed  by  5  x  3. 
Total  of  six  patterns  including  two  patterns  perpendicular  to  the  direction  of  a  muscular  fiber  and  parallel  at  each  joint  angle  (30°,  60°, 
90°)  was  measured.  For  verification,  we  measured  EMG  signal  nine  times  with  3x3  EMG  sensor  array  as  shown  in  Figure  7.2. 


(a)  The  way  a  tile  electrode  should  stick 


(b)  Torque  sensor 


Figure  7.5  Measurement  apparatus  used  in  experiment 


Table  2  Correlation  coefficient  of  EMG  map 


(a)  Parallel  to  a  muscular  fiber 
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Pattern  2 

0.82 

0.99 

0.72 

Pattern  3 

0.53 

0.69 

0.99 

(b) Perpendicular  to  a  muscular  fiber 


Verification  data 

Teaching  data 

Pattern  1 

Pattern  2 

Pattern  3 

Pattern  1 

0.99 

-0.48 

0.32 

Pattern  2 

-0.49 

0.99 

-0.33 

Pattern  3 

0.34 

-0.35 

0.99 

As  shown  in  Table  2,  the  correlation  coefficients  are  high  at  the  time,  when  the  pattern  of  teaching  data  and  verification  data  has 
overlapped.  From  this  result,  the  effectivity  of  EMG  tendency  map  for  the  purpose  of  presumption  of  a  position  error  is  confirmed. 


7.4  Summary 

An  auto -calibration  system  for  EMG  sensor  suit  has  been  developed.  This  developed  sensor  suit  has  tile-like  EMG  sensors.  Using 
this  tile-like  sensor  array,  the  innvertion  zone  in  human  body,  where  is  the  proper  position  for  EMG  sensors  to  measure  EMG  signal 
can  be  detected  correctly.  The  applicability  of  this  auto- calibration  system  is  confirmed  through  the  torque  estimation  experiment. 
Now  we  can  combine  this  system  with  our  sensor  suit. 


8.  Conclusion 

An  innovative  sensor  suit  is  developed,  which  can  be  conveniently  put  on  by  an  operator  to  detect  his  or  her  motion  intention  by 
non-invasively  monitoring  his  or  her  muscle  conditions  such  as  the  shape,  the  stiffness  and  the  density.  This  sensor  suit  is  made  of 
soft  and  elastic  fabrics  embedded  with  arrays  of  MEMS  sensors  such  as  muscle  stiffness  sensor,  ultrasonic  sensors,  EMG  sensors, 
accelerometers  and  optical  fiver  sensors,  to  measure  different  kinds  of  human  muscle  conditions.  In  this  project  period,  we 
successfully  developed  the  sensor  suit,  which  can  estimate  operator’s  torque  using  tile-like  EMG  sensor  array  with  an  auto-calibration 
system.  Moreover,  the  other  sensors  for  measurement  of  human  body  condition  including  the  muscle  stiffness  sensor,  the  muscle 
density  sensor,  the  MEMS  accelerometer  and  the  fiber  optic  sensor  have  been  successfully  developed  in  this  project  and  their  accuracy 
and  reliability  demonstrated.  We  had  also  demonstrated  broad  potential  applications  of  these  sensors,  particularly  in  the  biomedical 
field.  As  an  example,  we  presented  a  device  for  assisting  grasping  function  using  the  muscle  stiffness  sensor  and  the  sensor  network 
developed  in  this  project. 
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b.(5) 


Interactions  with  medical  field: 

1.  Device  for  assisting  grasping  function  using  muscle  stiffness  sensor 

2. Rehabilitation  system  of  hand  manipulation  using  optical  fiber. 

3. Communication  device  for  serious  disabled  using  MEMS  accelerometer 

4. Wearable  sensor  network  system  for  walking  assistance 


